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ABSTRACT 


Program Managers (PMs) throughout the Department of Defense (DoD) were 
directed by the DoD Chief Information Officer to manage information technology (IT) 
investments as portfolios (to include Mission Areas, Subportfolios, and Components) 
within the DoD Enterprise. Managing portfolios of capabilities aligns IT with the overall 
needs of the warfighter, as well as the intelligence and business activities which support 
the warfighter. This thesis provides the detailed steps that PMs and Program Executive 
Officers (PEOs) should follow to closely manage their IT portfolios using the concepts 
described within Harry Markowitz’ Modem Portfolio Theory. The first section will 
provide a demonstration of allocating revenue generated by a fictitious large corporation 
to the various sub-corporate levels and then applying Knowledge Value Added (KVA) in 
order to calculate a Return on Investment (ROI). The foundation of KVA analysis is that 
each subprocess output must be represented in common units of change; a price per unit 
of output is generated to allocate both cost and revenue at the subprocess level. The final 
section will apply a similar KVA analysis to the Naval Cryptologic Carry On Program 
(CCOP) systems to provide a public sector example. 
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I. INTRODUCTION 


A, PURPOSE / PROBLEM STATEMENT 

On October 30, 2006, the Department of Defense (DoD) Chief Information 
Offieer (CIO) direeted that all information teehnology (IT) investments must be managed 
as a portfolio. “Managing portfolios of eapabilities aligns IT with the overall needs of the 
warfighter, as well as the intelligenee and business aetivities whieh support the 
warfighter” (DoD, Information technology portfolio management implementation, 2006, 
p. 3). This direetive used some of the key terminology of Harry Markowitz’ Modern 
Portfolio Theory without actually naming it specifically. The DoD CIO did not provide 
speeifie guidanee on how to manage an IT investment portfolio, but only the neeessity to 
manage the portfolio within a non-profit organization, i.e., the DoD. The objeetive of 
this researeh is to eonduet a KVA analysis of a large eorporation and a non-profit military 
organization. This paper will provide a detailed framework in order to adhere to this 
direetive and will supply Program Managers with the neeessary toolset to be able to make 
informed IT investment deeisions. 

The Clinger-Cohen Act (also known as Information Technology Management 
Reform Aet of 1996) was signed into law in February 1996. The primary foeus of this 
doeument was aimed at ensuring the hundreds of federal ageneies (DoD, Federal Bureau 
of Investigation, National Seeurity Ageney, State Department, ete.) operated as 
eeonomieally responsible as possible. Aequisition, planning and management of IT must 
be treated as a “eapital investment.” This means that hardware and software eannot be 
purchased on an “impulse purehase” basis and installed without an overall plan. The 
Direetor of the Office of Management and Budget (0MB) is responsible for improving 
the aequisition, use, and disposal of IT by the Federal Government. His main goals were 
to better manage teehnology spending, aehieve measurable improvements in federal 
ageneies performanee, and control system development risks. 
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According to the Office of Management and Budget (2000), the Clinger-Cohen 
Act was established to improve the way Federal agencies acquire and manage their IT 
resources by: 

• Focusing information resource planning to support their strategic 

missions; 

• Implementing a capital planning and investment control process that 

links to budget formulation and execution, and 

• Rethinking and restructuring the way they do their work [(e.g., current 

processes)] before investing in information systems, (p. 2) 

“Consistent with the Office of Management and Budget (0MB) Capital Planning 
and Investment Control guidance, the Department of Defense shall use four continuous 
integrated activities to manage its portfolios” (DoD, Information technology portfolio 
management implementation, 2006, p. 3). Figure 1 illustrates these four components of 
managing IT investments within the DoD. 

• Analysis - “Activity in which Mission Areas and Subportfolios, in 
collaboration with Components, establish performance goals; identify gaps and 
opportunities; assess risks; provide for continuous improvement; and explore functional 
and technical options as documented in current capabilities and future integrated 
architectures” (DoD, Information technology portfolio management implementation, 
2006, p. 14). 

• Selection - “The process for selecting the best mix of investments to realize 
capabilities and achieve goals, satisfy measures, and comply with integrated architectures 
and transition plans” (DoD, Information technology portfolio management 
implementation, 2006, p. 18). The selection process focuses on a portfolio of IT 
investments with demonstrated ROI, benefit-cost analysis, proper planned oversight 
mechanisms, maximum usefulness, and other qualities aligned with the Enterprise 
Architecture (EA) and strategic plans as part of a managed investment portfolio. 

• Control - “Activity is focused on developing and acquiring the capabilities 
selected in the portfolio... The purpose of control is to ensure that these capabilities will 
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be acquired in conformance with Mission Area and Subportfolio expectations ...” (DoD, 
Information technology portfolio management implementation, 2006, p. 23). 

• Evaluation - The results of the investment after implementation to assess the 
benefit-cost achieved compared to the benefit-cost expected, evaluate the ROI to make a 
continue/modify/terminate decision on continuing with the investment, document lessons 
learned, redesign processes where needed, reassess the business case, technical 
compliance and EA compliance. 
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Eigurc 1. IT Portfolio Management Decision Support Interactions. (Erom: DoD, 
Information technology portfolio management implementation, 2006, p. 4) 

In the spring of 2006, the Department of the Navy created the Program Executive 
Office for Enterprise Information Systems (PEO-EIS). The PEO-EIS primary 
assignment was designed specifically to oversee a portfolio of enterprise-wide IT 
programs. Their mission statement is to develop, acquire and deploy enterprise-wide 
information technology systems with full life-cycle support for the warfighter and 
business enterprise. All of the PEO-EIS programs are specifically designed to maximize 
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value to the warfighter by balancing cost with the capability delivered to the end user. 
The following is the list of programs which have been developed, implemented and are 
currently being managed by PEO-EIS: 

a. BEII / ONE-Net: Base Eevel Information Infrastructure Program / OCONUS 
Navy Enterprise Network 

b. NMCI: Navy Marine Corps Intranet 

c. TEARS: Total Eorce Authorizations and Requirements System 

d. NSIPS: Navy Standard Integrated Personnel System 

e. Navy ERP: Navy Enterprise Resource Planning 

f. GCSS-MC: Global Combat Support System - Marine Corps 

g. Sea Warrior 

h. NM&P: Navy Manpower and Personnel Systems 

B, BACKGROUND 

This thesis represents the continuation of the work from two previous theses by 
ECDR Cesar Rios, in concert with Dr. Tom Housel in his thesis titled, “Return on 
Investment Analysis of Information Warfare Systems,” and ET Ira Eambeth/ET Hubert 
Clapp in their thesis titled, “Using Knowledge Value Added (KVA) for Evaluating 
Cryptologic IT Capabilities: Trial Implementation.” Both of these theses were conducted 
at the Naval Postgraduate School (NPS) in order to demonstrate the functionality and 
usability of the KVA methodology. Eambeth and Clapp conducted an operational 
example of six Navy Cryptologic Carry-On Program (CCOP) systems carried onboard 
the USS GONZAEES (DDG 66). 

ECDR Rios’ thesis focused on creating a foundation for analyzing the CCOP 
system using KVA methodology. The research conducted by ET Eambeth and ET Clapp 
centered on providing a proof of concept example of several CCOP systems installed 
onboard USS GONZAEES (DDG 66). The KVA methodology was applied to the CCOP 
systems used during an 18-month deployment of the USS GONZAEES. 
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This thesis is the follow-on research which will provide the detailed steps that 
Program Managers and Program Executive Officers should follow to closely manage 
their IT portfolio using the concepts described within Harry Markowitz’ Modem 
Portfolio Theory. The KVA methodology will be applied to two different companies 
(i.e., a for-profit large corporation and a non-profit military organization). The following 
background information will describe each company in further detail and the various 
concepts that will be applied throughout this thesis. 

• XYZ TELCOM: 

XYZ TELCOM is a fictitious telecommunication company based in Los Angeles 
with over 160,000 employees worldwide. There primary U.S. operating areas are 
California, Nevada, Texas, Kansas, Arkansas, Missouri, Michigan, Ohio, Illinois, 
Indiana, Wisconsin and Connecticut. This company is one of the largest providers of 
local and long distance telephone services in the United States. This company is fictional 
and is designed simply to be a proof of concept example to show that the KVA 
methodology can be used on any private sector company regardless of level of output or 
type of business organization (e.g., large corporation, sole-proprietor, partnership, etc.). 
KVA analysis can be performed on any company regardless of size or financial status. 
Using the KVA methodology, the Program Manager for XYZ TELCOM will be able to 
calculate the Return on Investment (ROI), Return on Knowledge (ROK) and the standard 
deviation of the IT systems in all the departments within the company. The steps detailed 
in this example will encapsulate the required tools necessary to be able to effectively 
manage any IT investment portfolio. 

• USS NEVERSAIL (DDG-22) Cryptologic Carry-On Program: 

The USS NEVERSAIL (DDG-22) is a fictional guided missile destroyer stationed 
in San Diego, CA. The ship carries six CCOP systems onboard during a routine six- 
month deployment. “The Cryptologic Carry-On Program (CCOP) is a product of the 
Advanced Cryptologic Systems Engineering program, which develops state-of-the-art 
Intelligence, Surveillance and Reconnaissance (ISR) capabilities in response to 
Combatant Command requirements for a quick-reaction surface, subsurface and airborne 
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cryptologic carry-on capability” (Rios, 2005, p. 2). CCOP supports Fleet Commander in 
Chief (FLTCINC) cryptologic Electronic Support (ES) requirements for Naval and Joint 
Operation including Counter-Drug operations. Its manning is provided by fleet Direct 
Support (DIRSUP) Augmentees and is staged at Eleet Electronic Support (EES) activities 
worldwide. 

The CCOP systems are composed of several different subsystems, which for 
classification purposes will be referenced simply by a letter. The specifics of the system 
are outside the scope of this paper in order to keep this thesis at an unclassified level. All 
of the data presented is fictitious and will be used as a proof of concept example to 
illustrate how KVA analysis can be performed on any non-profit organization regardless 
of size. 

The following concepts will be applied throughout this thesis: 

1. Return on Investment 

Return on Investment (ROI), also known as rate of return, is the ratio of money 
gained or lost on an investment relative to the money invested. To calculate ROI, the 
benefit (return) of an investment is divided by the cost of the investment; the result is 
expressed as a percentage or a ratio. ROI is a basic productivity ratio with profit 
(revenue minus cost) in the numerator and cost to generate the profit in the denominator. 


Percentage ROI = from Investment - Cost of Investment) 

Cost of Investment 

Eor the above formula, the numerator is the difference between revenue and cost 
of the investment (or asset) and the denominator represents the cost of the investment (or 
asset) to the organization. “The ROI then produces a metric to determine how efficiently 
the investment is applied” (Eambeth & Clapp, 2007, p. 4). A positive value corresponds 
to capital growth, a negative value corresponds to capital decay and a value of 0% 
corresponds to no change. 
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Referring to revenues throughout this paper may seem imprudent due to the fact 
that the U.S. military is not a profit making business. This is the primary reason why the 
KVA methodology was chosen for this research. For non-profit organizations, especially 
in the military, Knowledge Value Added (KVA) is utilized to provide the required 
“benefits” or “revenue” proxy estimates to run ROI analysis. KVA generates ROI 
estimates by developing a market comparable price-per-common-unit of output 
multiplied by the number of outputs to achieve a total revenue estimate. 

2. Knowledge Value Added 

“The Knowledge Value Added (KVA) methodology was created by Drs. Thomas 
Housel and Valery Kanevsky and has been published internationally in numerous articles 
and books about knowledge management and business process reengineering” (Housel & 
Kanevsky, n.d, p. 1). The KVA methodology provides a means to objectively allocate 
revenue to all corporate resources (e.g., materials, supplies and equipment) and 
organizational processes, even down to the individual employee level. It is based on the 
premise that businesses and other organizations produce outputs (e.g., products and 
services) through a series of processes and subprocesses which change the raw inputs 
(e.g., labor into services, information into reports). 

Housel and Bell (2001) defined KVA as the following: 

KVA is a methodology designed to estimate the value of the knowledge 
deployed throughout a company’s core processes. KVA provides a 
performance ratio estimate that standardizes the output of all processes by 
describing the output in terms of the units of knowledge required to 
produce the output. 

The goal of KVA methodology is to describe all organizational outputs in 
common units. Common units provide a means to compare all of the outputs for the 
organization’s assets (e.g., human, machine, IT) regardless of the total quantity of outputs 
produced. An example within the military would be the process of planning for a ship 
alteration or gathering signal intelligence. KVA would allow a Program Manager to 
describe the outputs of both processes in common units and provide him the ability to 
compare their performance equally. 
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According to Housel & Mun (n.d.): 


KVA measures the value provided by human capital assets and IT assets 
by analyzing an organization, process or function at the process level. It 
provides insights into each dollar of IT investment by monetizing the 
outputs of all assets, including intangible assets (e.g., such as those assets 
produced by IT and humans). By capturing the value of knowledge 
embedded in an organization’s core processes (i.e., employees and IT), 

KVA identifies the actual cost and revenue of a process, product, or 
service. Because KVA identifies every process required to produce an 
aggregated output in terms of the historical prices and costs-per-unit of 
output of those processes, unit costs and unit prices can be easily 
calculated. 

As a performance tool, the KVA methodology: 

• Compares all processes in terms of relative productivity 

• Allocates revenues and costs to common units of output 

• Measures value added by IT by the outputs it produces 

• Relates outputs to cost of producing those outputs in common units. 

(p. 3) 

3, Modern Portfolio Theory 

“In 1952, Harry Markowitz revolutionized the field of finance with his article 
written in the Journal of Finance entitled ‘Portfolio Selection’” (Markowitz, 1987, p. ix). 
Modern Portfolio Theory (MPT) considers a method of resource allocation for the 
investor in the for-profit sector. Markowitz introduced the idea of a mean-variance 
efficient portfolio as one that (1) provides minimum variance for a given expected return 
and (2) provides maximum expected return for a given variance. The main idea is that 
investors should maintain a well-diversified portfolio in order to reduce their overall risk. 
“A ‘portfolio’ is a list of securities that belongs to an individual investor or a group of 
investors having certain goals” (Smith, 1971, p. 40). 

MPT starts with the investors’ set of probability beliefs regarding the expected 
return from each investment and the expected covariance between each pair of 
investments (based in turn upon expected standard deviations and correlation 
coefficients). Given these probability beliefs, the investor can chose between various 
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combinations of reward (expected return) and risk (variance of returns) depending on the 
construction of the portfolio (the identity and relative proportions of the investments). Of 
these combinations, those with the minimum variance for a given (investor-determined) 
level of risk or maximum return for a given variance correspond to a set of “efficient” 
portfolios. Thus, portfolio theory suggests a way of optimally allocating capital for the 
investor in the private sector. 

Mclnish (2000) wrote in his book entitled ""Capital Markets: A Global 
Perspective 

A portfolio is efficient if it has the highest return for a given level of risk 
and the lowest risk for a given level of return. One portfolio dominates 
another if it has more return for a given level of risk or less risk for a given 
level of return, (p. 287) 

Some PMs measure the amount of risk that their firm will tolerate. Then they 
examine a list of trade offs between more reward and lower risk. One of the components 
of MPT is the figure shown below which shows how much risk investors are willing to 
accept for a given return. If they are given an investment with a high potential return, 
investors are usually willing to accept a little more risk. If they are given an investment 
with much more certainty, they are willing to accept a lower return. This is expressed as 
a curve on the chart where risk and return are just barley acceptable. Figure 2 illustrates 
what someone’s investment boundary might look like. 
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Figure 2. The Investment Boundary. (After: Hubbard, 2007, p. 192) 

This is a little different from the chart Harry Markowitz used. His risk axis was 
really historical volatility of the return on a particular stock (capital gains of losses as 
well as dividends). Markowitz’s theory is an excellent tool for investors to use in order 
to be able to manage their investment portfolio. Modern Portfolio Theory is designed 
specifically for an investment portfolio of the stock market; however the principles can 
easily be tailored to work with any IT investment portfolio. In the stock market, the risk 
involved in a particular stock is determined by the historical volatility of the return. “But 
investments like IT projects or new product development don’t typically have ‘historical 
volatility.’ They do, however, share another characteristic of risk that is more 
fundamental than Markowitz’s measure: They have a chance of a loss” (Hubbard, 2007, 
p. 193). 
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4, Capital Asset Pricing Model 


According to Mclnish (2000): 

The Capital Asset Pricing Model is a major extension of the work of 
Markowitz. The CAPM has proved useful in a variety of contexts, 
including estimating the cost of capital, estimating risk for portfolios, and 
developing measures for ex post portfolio performance evaluation. The 
CAPM has also given rise to a significant movement to passively invest in 
portfolios designed to mimic indexes rather than trying to beat the market, 
(p. 281) 


The CAPM is a model for pricing an individual security or a portfolio. The model 
is used to determine a theoretically appropriate required rate of return of an asset by 
taking into account the asset’s sensitivity to non-diversifiable risk, the expected return of 
the market and the expected return of a theoretical risk-free asset. 

The following excerpt was taken from “Capital Asset Pricing Model” (n.d.): 


For individual securities, we made use of the security market line (SML) 
and its relation to expected return and systemic risk (beta) to show how 
the market must price individual securities in relation to their security risk 
class. The SML enables us to calculate the reward-to-risk ratio for any 
security in relation to that of the overall market. Therefore, when the 
expected rate of return for any security is deflated by its beta coefficient, 
the reward-to-risk ratio for any individual security in the market is equal to 
the market reward-to-risk ratio, thus: 


E(Rd - R, 
A 


— E{Rjn) — Rf 


The market reward-to-risk ratio is effectively the market risk premium and 
by rearranging the above equation and solving for E(Ri), we obtain the 
Capital Asset Pricing Model (CAPM). 


E(R,) = Rf + 3i(E(R,„)-Rf) 


Where: 


E(Rd= 

f — 


the expected return on the capital asset 


^f = the risk-free rate of interest such as interest arising from 
government bonds 

= the sensitivity of the asset returns to market returns. 
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rn/ = the expected return of the market 


5. Tracking Stocks 

In the late 1990s, several companies were conducting research on ways to track 
the value of one segment of the organization. Many companies issued “tracking” stocks 
in addition to their traditional common stock. “Tracking stock, also known as ‘targeted’ 
or ‘lettered’ stock in the financial press, is a class of common stock that is linked to the 
performance of a specific business group within a diversified firm” (Elder & Westra, 
2000, p. 1). A tracking stock was a type of common stock that “tracks” or depends on the 
financial performance of a specific business unit or operating division of a company. The 
advantage of this type of stock was based on the concept that when the unit or division 
does well then the value of the tracking stock may increase even if the company as a 
whole performs poorly. 

According to Elder & Westra, companies like Walt Disney Co., AT&T and Sprint 
Corporation are some of the few companies who decided to issue tracking stocks. These 
companies attempted to track the value of one particular division within their large 
organization. Unfortunately there were several disadvantages associated with tracking 
stocks such as: (1) the lack of a separate board of directors to oversee the division; (2) 
tracking stocks tended to perform about as well as the company’s common stock; (3) 
tracking stock shareholders usually have limited or no voting rights. Based on the poor 
performance of tracking stocks over the years there are very few tracking stocks in 
existence today. However, the concept of allocating and tracking the value of the sub¬ 
corporate levels within an organization is highly sought after by most companies and of 
significant relevance to this paper. 

C. RESEARCH OBJECTIVES 

The objective of this research is to conduct a KVA analysis on a for-profit large 
corporation and a non-profit military organization. This thesis will build upon previous 
research to provide the detailed framework that Program Managers and Program 
Executive Officers should follow to closely manage their IT portfolio. The fundamental 
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concepts which are described within Harry Markowitz’ Modem Portfolio Theory will be 
the foundation for this research. The goal of this research is to provide PMs the 
necessary toolset to make informed IT investment decisions and effectively manage their 
IT projects. 

D, METHODOLOGY 

The Knowledge Value Added (KVA) methodology will be used to analyze two 
different types of companies. The first KVA analysis will be conducted on a fictional 
large corporation named XYZ TELCOM which is designed to represent any private 
sector company. The second analysis will be conducted on the Navy’s Cryptologic 
Carry-On Program carried onboard the fictitious USS NEVERSAIL (DDG 22) during a 
routine six-month deployment. These two analyses are proof of concept examples in 
order to demonstrate the various steps involved with KVA. The KVA methodology can 
be easily tailored to fit a non-profit organization such as the U.S. government or a private 
sector company. The cost and price per unit of output will be estimated using the KVA 
methodology which describes all outputs in common units. The benefit to a PM of 
utilizing the KVA analysis is the fact that the final results will produce a Return on 
Investment (ROI) value and a standard variation percentage which can be used to manage 
a portfolio of various IT investments. 
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II. KVA ANALYSIS XYZ TELCOM 


A, INTRODUCTION 

The fictitious XYZ TELCOM company provides telecommunications services to 
consumers and businesses in the United States and internationally. It provides wireless 
services, including local wireless communications, long-distance, and roaming services 
with various postpaid and prepaid service plans. The company also supplies various 
handsets and personal computer wireless data cards, as well as accessories comprising 
carrying cases, hands-free devices, batteries, battery chargers, and other items. XYZ 
TELCOM’s wireline services comprise voice services comprising local and long-distance 
services, integrated network connections, traditional long distance and international long 
distance, calling card, 1-800 services, conference calling, and wholesale switched access 
service, as well as calling features, such as caller ID, call waiting, and voice mail. Its 
wireline data services consist of switched and dedicated transport, Internet access and 
network integration, and data equipment. The company's other wireline services include 
managed Web hosting, application management, security service, integration services, 
customer premises equipment, outsourcing, directory and operator assistance services, 
government-related services, and satellite video services. 

B, APPLYING KVA METHODOLOGY 

1. KVA Assumptions 

According to Rios, Housel & Mun (2006): 

Based on the tenets of complexity theory, KVA assumes that humans and 
technology in organizations add value by taking inputs and changing them 
(measured in units of complexity) into outputs through core processes. 

The amount of change an asset produces within a process can be a 
measure of value or benefit. 

The additional assumptions in KVA include: 

• Describing all process outputs in common units (e.g., using a 
knowledge metaphor for the descriptive language in terms of the time it 


15 



takes an average employee to learn how to produce the outputs) allows 
historical revenue and cost data to be assigned to those processes 
historically. 

• All outputs can be described in terms of the time required to learn how 
to produce them. 

• Learning Time, a surrogate for procedural knowledge required to 
produce process outputs, is measured in common units of time. 
Consequently, Units of Learning Time = Common Units of Output (K). 

• A common unit of output makes it possible to compare all outputs in 
terms of cost-per-unit as well as price-per-unit, because revenue can now 
be assigned at the sub-organizational level. 

• Once cost and revenue streams have been assigned to sub- 
organizational outputs, normal accounting and financial performance and 
profitability metrics can be applied, (p. 9) 

2. KVA Steps 

KVA steps should be followed by Program Managers and Program Executive 
Officers when managing their IT portfolios. By closely monitoring the IT portfolio in 
this manner it will provide the PM with greater insight into determining whether to 
continue to fund a particular system, upgrade the system or possibly even abandon the IT 
system altogether. These KVA steps encompass the combined works of Drs. Housel & 
Kanevsky (n.d.) and Lambeth & Clapp (2007). 

a. Step One: Identify Core Process and Its Subprocesses 

• Determine periphery of the core processes. 

• Achieve consensus on the description of the complex and basic processes. 

• Utilize current descriptions, found in diverse places (e.g., in historical records, 
activity based costing efforts or quality management efforts). 

• Apply any of the workflow modeling tools to generate a document that 
includes comprehensive examples of the core and subprocesses, to include process 
diagrams that have a clear input and output. 

• Explain the tasks for the subprocesses and core process. 
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b. Step Two: Determine Learning Time 


Learning Time (Lt) is the time required for the average worker to learn 
how to produce a single process output is estimated. 

c. Step Three: Determine How to Execute a Process 

Executions (Ex) are the number of times all process assets, human or IT, 
produced the process outputs in a given time-period. Compute the amount of process 
commands that are essential to generate the outputs of each process. 

d. Step Four: Calculate Value of Output (K) and Total Value of 
Output (K)for each Process 

Knowledge (K) is the term used to describe common units of output 
estimated by KVA. The formula for calculating knowledge is to multiply the executions 
by the learning time. K = (Ex) (Lt). 

Calculate the sum of K for each of the sub-processes, the result will be the 
Total K for the sub-processes. Total K = L (K). 

e. Step Five: Determine Revenue per Knowledge Unit and Establish 
the Numerator by Assigning Revenue Streams to Processes 

Revenue per Knowledge Unit (RKU) is the price per unit designated for 
the outputs for the profit organizations based on the market price per unit of the 
comparable outputs of a similar commercial organization. 

Revenue (R) is allocated to each process by multiplying the common 
units of output (K) of each process by the RKU. R = (RKU) (K). 

f Step Six: Calculate Process Cost 

The expenses show the cost associated with producing the output of each 
process. Sum the costs of assets in each process to yield the total costs of all outputs of a 
process for a given time period. 
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g. Step Seven: Calculate the Value Ratios ofROI and ROK 

“As knowledge initiatives take hold and mature, it becomes essential to 
conduct true cost/benefit analysis to determine the proper level of investment for 
knowledge management and the expected return on investment” (“Measuring the Return 
on Knowledge,” 2006). The Return on Knowledge (ROK) provides a ratio of how the 
cost and revenue are associated with producing an output within each sub-process. The 
ROK is calculated by dividing the revenue by the expenses. ROK can be compared 
among processes to determine which processes may need closer analysis when the result 
is a low ROK. ROK = Revenues / Expenses, 

The Return on Investment (ROI) is a return ratio that compares the net 
benefits of a process verses its total costs. The ROI is calculated by the taking the 
numerator minus the denominator and then dividing the resultant by the denominator. 
ROIs can be compared among processes to determine which processes may need to be 
changed in order to improve efficiency. ROI = (Revenues - Expenses ) / Expenses, 

h. Step Eight: Calculate Standard Deviation (If Applicable). 

Calculating standard deviation provides the PM with an indication of the 
volatility of an asset. Volatility is a degree of uncertainty that exists in a program. The 
steps are only applicable if historical ROI data is present. Standard deviation is the 
square root of the variance and can be derived by the steps indicated below: 
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Population Mean (//) 


M = 



N 


// is the average value of the population. 

^ is the total of or sum of variable 'x' 

X is the values of the Historical ROIs ( Ql, Q2,....) 
N is the number of values or data points 


Variance (cr^) 

N 


Standard Deviation (a) 


(j = 



i. Calculate the Population Mean 

1. Add the values of the historical ROIs. 

2. Divide the sum of historical ROIs by N. 

ii. Calculate the Variance 

1. Subtract the mean from each variable of A’. 

2. Square each result. 

3. Sum all the squares. 

4. Divide the sum of squares by N. The result is the variance. 

iii. Calculate the Standard Deviation 

1. Square root of the variance. 
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C. KVA RESULTS 


Department 

Learning 

Time 

Executions 

Knowledge 

(K) 

Expenses 

Revenues 

ROK 

ROI 

STD 

DEV 

% of Rev. 

Accounting 

15 

800,000 

12,000,000 

$ 1,100,000 

$ 1,200,000 

109.09% 

9.09% 

4.0% 

12.00% 

Human 

Resources 

10 

1,000,000 

10,000,000 

$ 900,000 

$ 1,000,000 

111.11% 

11.11% 

2.8% 

10.00% 

Manufacturing 

0.5 

110,000,000 

55,000,000 

$ 5,000,000 

$ 5,500,000 

110.00% 

10.00% 

2.6% 

55.00% 

IT 

30 

500,000 

15,000,000 

$ 1,200,000 

$ 1,500,000 

125.00% 

25.00% 

7.6% 

15.00% 

Sales 

5 

1,600,000 

8,000,000 

$ 650,000 

$ 800,000 

123.08% 

23.08% 

6.4% 

8.00% 

Totals 



100,000,000 

$ 8,850,000 

$ 10,000,000 

112.99% 

12.99% 

3.84% 

100.00% 

Revenue per 
Knowledge Unit: 

$0.10 










Table 1. KVA Analysis for XYZ TELCOM 


Table 1 illustrates the implementation of using the KVA methodology on five 
major processes: Accounting, Human Resources, Manufacturing, Information 
Technology and Sales. The results of the KVA analysis for each of these processes for 
XYZ TELCOM are shown in Table 1. This analysis makes possible the allocation of IT 
resources based on the amount of knowledge embedded within each segment’s core 
processes. Given a periodic application of the KVA methodology, the company can 
build an historical record of sub-corporate revenues and arrive at a variance (shown as 
standard deviation) and correlations between the earnings of the various segments. In 
effect, the PM has all the information needed to extend the notion of portfolio theory as 
applied to investments in companies down to a sub-corporate market. 

D. ANALYZING KVA RESULTS 

The KVA analysis produces a Return on Knowledge (ROK) which is a ratio to 
estimate the value added by given knowledge assets. By comparing the expenses and 
revenues associated with the knowledge asset, the results can be computed to compare 
efficiency in performance of the processes. In the example in Table 1, XYZ TELCOM’s 
ROK for each of the five processes generated positive net revenue, which is indicated by 
an ROK of greater than 100%. 


20 



















ROI can be used by a Program Manager or a Program Executive Officer to 
determine whether current or prospective IT investments should be funded or changed in 
order to improve efficiency. According to the Department of the Navy’s IT Capital 
Planning Guide (2001), investments must have an ROI greater than “1.0” to be 
considered for funding. In this example, Table 1 shows that all of the core processes for 
XYZ TELCOM have an ROI greater than 1.0. Therefore the PM would make favorable 
recommendations based on the ROI results for the IT investment. Periodic applications 
of the KVA methodology can provide a PM with a historical record of an IT assets past 
performance. The PM can evaluate the major segments’ ROI to determine whether to 
reallocate resources based on his analysis. 

At this point in the ROI process, the Program Manager’s recommendations are 
heavily influenced by the historical data that has been collected over a measured period 
of time. It is the recommendation by the authors of this thesis that PMs collect and 
analyze quarterly ROI data in order to have sufficient historical data to better determine 
volatility. The PM assigns the weights to each metric and then he determines if the 
metric is following a positive or negative trend. It is imperative that the PM ascertains 
solid metrics before making an investment decision so that he can better analyze the ROI 
process. If the historical data is not present, then the PM can still define the metric based 
on the current ROI analysis. 

The next step is to use real options analysis to assess the IT processes of XYZ 
TEECOM. According to Mun (2006), real options is defined as “a systematic approach 
and integrated solution using financial theory, economic analysis, management science, 
decision sciences, statistics, and economic modeling in applying options theory in 
valuing real physical assets ...” 
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Figure 3 is a list of the various types of options and their industry applieations. 


Types of Options 

Industry Applications/Users 

• Option to Wait 

• DoD/Acquisitions, Eorce Mix 

(Proof of concept, right of first refusal, getting more info) 

• CCOPS Portfolio Analysis 

• Option to Execute 

• Aeronautics/Boeing, Airbus 

(Contracts in place which may/not be executed) 

. Oil and Gas/BP, Shell 

• Abandonment Option 

• High Tech/Intel 

(exit and salvage) 

• Pharmacology/Merck, Pfizer 

• Expansion Option 

• R&D Portfolios/Motorola, 

(platform technologies, acquisitions, open architecture) 

Unilever 

• Contraction Option 

• IT Infrastructure/Credit Suisse 

(outsourcing, alliances) 

• Electricity/Peaker-Plants 

• Compound Option (platform options) 

• Acquisitions/Seagate 

• Sequential Options (stage-gate development, R&D, phased 
options) 

• Contracts/Syngenta, GM 


Figure 3. Types of Real Options and Industry Applieations (From: Rios, Housel, and 

Mun, 2006) 


The eight steps of the real-options approaeh are illustrated in Figure 4. 


1. List of projects and 
strategies to evaluate 



. Base case projections for each 
project 


Historical Data Gathering 
Time Series Forecasting 




Start with a list of projects or 
strategies to be evaluated... these 
projects have already been 
through qualitative screening 


5. Framing Real Options 



..^LlllUlceLlOLl 


3. Develop static financial 
models with ICVA data 


. .will the assistance of time- 
series forecasting and 
historical data... 


6. Options analytics, simulation 
and optimLsation 



4. Dynamic IMontc Carlo 
simulation 


Simulation 

Lognormal 


series of static base case financial 
(discounted cash flow) models for 
each project... 


...sensitivity and scenario analysis 
coupled with Monte Carlo simulation is 
added to the analysis and the financial 
model outputs become inputs into the rea 
options analysis... 


. Portfolio optimization and asset 
allocation 


8. Reports presentation and 
update analysis 


.. .the relevant projects art 
chosen for real options 
analysis and the project oi 
portfolio real options are 
framed... 


Closed-Form Models 


...real options analytics are calculated 
through binomial lattices and closed-form 
partial-differential models with 
simulation... 



.. .stochastic optimization is the next 
optional step if multi ple p rojects exist that 
requires efficient asset allocation given 
some budgetary constrainLs... useful for 
strategic portfolio management... 


. .create reporLs, make decisions, 
and do it all again iteratively over 


Figure 4. Real Options Analysis Steps (From: Mun, 2006, p. 29) 
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According to Mun (2006), real-options can be used to: 

• Identify different corporate investment decision pathways or projects 
that management can navigate given the highly uncertain business 
conditions; 

• Value each of the strategic decisions pathways and what it represents 
in terms of financial viability and feasibility; 

• Prioritize the pathways or projects based on a series of qualitative and 
quantitative metrics; 

• Optimize the value of your strategic investment decisions by 
evaluating different decision paths under certain conditions or determining 
how using a different sequence of pathways can lead to the optimal 
strategy; 

• Time the effective execution of your investments and finding the 
optimal trigger values and cost or revenue drivers; and 

• Manage existing or developing new optionalities and strategic decision 
pathways for future opportunities. 


Due to the positive trends in the ROI analysis for Table 1, the PM for XYZ 
TELCOM should consider one or more of the following real options: option to wait; 
option to execute; or expansion option. The option to wait would probably be the best 
option due to the fact that the PM could use more historical ROI data to get a better 
assessment of the volatility of the IT portfolio. 

A PM can get real options implemented by understanding that real options 
analysis is a decision-making process that improves and balances the decision analysis 
approaches. Real options take financial methodologies and change them to analytical 
techniques. A PM must have the right tools, resources, and senior management buy-in to 
successfully implement a real options analysis. 
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III. KVA ANALYSIS USS NEVERSAIL (DDG-22) 


A. INTRODUCTION 

The first KVA analysis performed on a U.S. military system was conducted by 
Lieutenant Ira Lambeth and Lieutenant Hubert Clapp. Their focus was centered on 
performing a proof of concept study to utilize KVA methodology to measure the ROI of 
six different Cryptologic Carry-On Program (CCOP) systems. This methodology was 
applied to the CCOP systems in use during an 18 month deployment of the USS 
GONZALES (DDG 66). The ROI data was analyzed and modeled using GaussSoft KVA 
Performance Accounting Modeling Software. A detailed analysis was conducted on the 
performance of each CCOP system, each stage of the Intelligence Collection Process 
(ICP) and the individual operators themselves. The results served as inputs for analysis 
for decision makers to study alternative courses of action (COAs) for the future 
deployment of CCOP systems. 

Using this initial KVA analysis as a baseline, an example was created using a 
fictional guided missile destroyer USS NEVERSAIL (DDG-22). In this example, the 
ship has six CCOP systems carried onboard during a routine 6-month deployment. All of 
the data is merely used as a proof of concept example and the figures do not represent 
any real-world values. The analysis should be used by Program Managers and Program 
Executive Officers to better understand how to properly manage their IT investment 
portfolios. 

B. APPLYING KVA METHODOLOGY 

1, KVA Assumptions 

“KVA uses a knowledge-based metaphor as a means to describe units of change 
in terms of the knowledge required to make the changes” (Lambeth & Clapp, 2007, p. 
14). The fundamental assumptions of KVA have not changed and are exactly the same as 
the example in the previous chapter. Eigure 4 illustrates the underlying assumptions of 
KVA analysis. 
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Change, Knowledge, and Value are 
Proportionate 



P(X) = Y 


Fundamental assumptions: 

1. If X = Y no value has been added. 

2. "value" « (is proportionate to) "change" 

3. "change" can be measured by the knowledge required to 
make the change. 

So "value"oc "change" oc"knowledge required to make the 
change" 


Figure 5. Underlying KVA assumptions model (From: Cook, Slide 6). 


These original assumptions are further broken down in order to demonstrate the 
principle concept that KVA can be used on any type of organization even when no 
revenue is generated. These universal assumptions are: 


a. Cost and Revenue Assumptions 

Describing all process outputs in common units (e.g., using a knowledge 
metaphor for the descriptive language in terms of the time it takes an average employee 
to learn how to produce the outputs) allows historical revenue and cost data to be 
assigned to those processes historically. 

According to Lambeth & Clapp (2007): 

Having a common unit of output makes it possible to compare all outputs 
in terms of cost-per-unit as well as price-per-unit, since revenue can now 
be assigned at the sub-organizational level. Once cost and revenue 
streams have been assigned to sub-organizational outputs, normal 
accounting and financial performance and profitability metrics can be 
applied to them. (p. 15) 
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b. Learning Time 


There are inherent similarities and differences between knowledge and 
learning. Both may be explicit or tacit; however, learning time is more about “How” and 
less about the “Why”. All outputs can be described in terms of the time required to learn 
how to produce them. Knowledge and learning are defined differently: 

• Knowledge may be specific or general. 

• Knowledge may be in people or IT. 

• Learning is the process (time) to gain knowledge. 

• Learning is related to task specific activities. 

• Learning Time is measured in common units of time. 

2. Case Study Assumptions and Data 
a. Assumptions 

Lambeth & Clapp (2007) wrote the following concerning market 
comparable revenue assumptions: 

The [market comparable] revenue assumption states that not for profit 
agencies can derive certain inferences from comparable outputs of 
commercial entities. They are: 

• First, if the processes used to produce the outputs of both organizations 
are comparable, then the outputs of the two must also be comparable. 

• Second, if market forces have placed a “value” or price-per-unit to the 
comparable commercial outputs yielding a revenue stream for the 
commercial entity, that price-per-unit can also be applied to the not-for- 
profit case. 

• Lastly, the derived price-per-unit can be used to develop an analytical 
or hypothetical revenue stream for the not-for-profit organization, (p. 15) 
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b. Data 

Length of Sample Period: The sample period for this analysis was the 
normal 6 month deployment period. Annual cost data have been adjusted to reflect this 
time frame. 

Learning Time: The learning time for this example is estimated and 
reflects how long it would take the average person to learn how to produce the outputs of 
each CCOP system. Learning time can be obtained by conducting a face-to-face 
interview with the senior leadership within the organization. A secondary interview can 
also be conducted with the actual users of the various systems and then comparing the 
results with the data obtained with the interview previously performed. However, be 
advised that achieving a consensus between top executives and the individual employees 
is often difficult to obtain. The users of the system sometimes do not fully comprehend 
all the subprocesses that comprise the total amount of learning time to produce the final 
output. Typically, senior leaders have greater insight and a broader understanding as to 
all of the various complexities involved in order to produce the required level of output. 
Learning time includes formal training (class work), on-the-job training and hands-on 
apprenticeship work. Learning time is “time spent learning” and not elapsed time. 

Executions: This value is a measure of the number of times a process 
executes or knowledge is used to perform a step in a process. The data for the USS 
NEVERSAIL (DDG-22) was manufactured for the purposes of simplifying the 
calculation process only and is not attributable to any actual real-world source. 

Cost and Revenue Assumptions: Detailed cost and revenue information 
was estimated for the USS NEVERSAIE CCOP systems. Cost data was derived from 
annual budget estimates. Revenue data was derived from market comparables to 
commercial units of the same common units of output as the six CCOP systems. Cost 
and revenue data is displayed in the following table: 
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Element 

Avg. Annual Unit Costs 

Avg. Annual Unit Revenue 

CCOP A 

$158,333 

$102,530 

CCOP B 

$29,167 

$103,154 

CCOPC 

$54,545 

$94, 072 

CCOP D 

$40,000 

$61,768 

CCOP E 

$35,000 

$53,033 

CCOP F 

$58,000 

$45,442 

TOTAL IT 

$330,898 

$460,000 


Table 2. USS NEVERSAIL Systems Cost and Revenue Data. 


3, KVA Results 

The KVA steps are detailed in Chapter II of this thesis. To simplify this example 
the steps will not be demonstrated. Program Managers must realize, however, that KVA 
analysis cannot be performed without first performing all the required steps and 
calculating all the necessary data. Table 3 contains the results of the KVA analysis for 
each of the six CCOP systems on USS NEVERSAIE (DDG 22). 


Element 

Learning 

Time 

Executions 

Knowledge 

(K) 

Expenses 

Revenues 

ROK 

ROI 

STD 

DEV 

%of 

Rev. 

CCOP A 

34 

87 

2958 

$ 158,333 

$ 

102,530 

64.76% 

-35.24% 

4.0% 

22.29% 

CCOP B 

93 

32 

2976 

$ 29,167 

$ 

103,154 

353.67% 

253.67% 

2.8% 

22.42% 

CCOPC 

59 

46 

2714 

$ 54,545 

$ 

94, 072 

172.47% 

72.47% 

2.6% 

20.45% 

CCOP D 

18 

99 

1782 

$ 40,000 

$ 

61,768 

154.42% 

54.42% 

7.6% 

13.43% 

CCOP E 

85 

18 

1530 

$ 19,833 

$ 

53,033 

267.40% 

167.40% 

12.0% 

11.53% 

CCOP F 

57 

23 

1311 

$ 29,000 

$ 

45,442 

156.70% 

56.70% 

6.4% 

9.88% 

Totals 



13271 

$ 330,878 

$ 

460,000 

139.02% 

39.02% 

3.70% 

100.00% 

Revenue per 
Knowledge Unit: 

$ 34.66 










Table 3. KVA Analysis for USS NEVERSAIE (DDG 22) 


C. ANALYZING KVA RESULTS 

The results of the USS NEVERSAIL KVA calculation are purely fictitious and 
are merely shown as a public sector example. The main focus should be placed on the 
ROI and Standard Deviation columns. These columns represent the amount of Return on 
Investment and the volatility (risk) for each of the six different CCOP processes. The PM 


29 






























would be able to make recommendations to the Navy PEO based on the various real 
options available. Most of the values contain positive ROI percentages, although the 
calculation of CCOP A resulted in a negative ROI. This negative ROI will need to be 
examined and analyzed further. 

There is a fundamental difference between managing an individual information 
technology (IT) portfolio and managing a large aggregated IT portfolio of several 
individual investments. A Program Manager in charge of a small IT portfolio is only 
concerned with his individual system(s), but a Program Executive Officer in charge of a 
large portfolio that encompasses several individual IT portfolios would be have a broader 
focus. The PM of the USS NEVERSAIE (DDG-22) is concerned about the negative ROI 
for CCOP A while also noticing that the standard deviation is 4.0% (meaning that the 
volatility of the system is low and relatively stable). An ROI less than 1.0 indicates a 
negative trend of an IT portfolio and therefore should be closely monitored by the PM. 
The PM realizes that due to the low volatility of CCOP A the ROI will remain negative 
without some type of process re-engineering. The Navy PEO in charge of all the CCOP 
systems has a broader focus. His job is to calculate the ROI data for every CCOP system 
in the entire Navy. He is not greatly concerned with the single USS NEVERSAIE 
portfolio, but is focused on the total ROI of the larger portfolio made up of all CCOPs in 
the Navy. Each PM must realize that there are two separate points of view that at times 
may differ from one another. 

The PM for the USS NEVERSAIE must never develop tunnel vision while 
managing his IT portfolio. A Program Manager’s overall goal is to effectively manage 
his IT portfolio to the utmost of his abilities. One of his primary concerns would be to 
reduce expenses as much as feasible while continuing to increase revenue thus creating a 
positive ROI. The term “revenue” in this example is not literal given that this is a public 
sector organization. An effective manager would also strive to reduce learning times to 
streamline the process. However, the PM needs to understand that his efforts to 
successfully manage his individual IT portfolio could have a positive or negative affect to 
the aggregated portfolio of the organization. 
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Using the USS NEVERSAIL as the primary example to illustrate these possible 
side effects, the PM notices that the calculation in the first row of Table 3 resulted in a 
negative ROI for CCOP A. This can be easily explained by the fact that the expenses 
greatly outweigh the revenue generated by the process. The PM immediately analyzes 
the data and determines solutions in order to produce a positive ROI. One of the possible 
viable solutions would be to increase the amount of executions of CCOP A in order to 
thereby increase revenue. In this particular case, a positive ROI would result when the 
number of executions was increased from 87 to 145. Even though the CCOP A process 
is 95% automated by computer there would still be an increase of $7,000 in expenses. 
Table 4 below contains the revised calculated totals that would result from this increase 
in executions of CCOP A system (changes are highlighted in yellow). 


Element 

Learning 

Time 

Executions 

Knowledge 

(K) 

Expenses 

Revenues 

ROK 

ROI 

STD 

DEV 

CCOP A 

34 

145 

4930 

$ 165,333 

$ 170,874 

103.35% 

3.35% 

4.0% 

CCOP B 

93 

32 

2976 

$ 29,167 

$ 103,154 

353.67% 

253.67% 

2.8% 

CCOPC 

59 

46 

2714 

$ 54,545 

$ 94,072 

172.47% 

72.47% 

2.6% 

CCOP D 

18 

99 

1782 

$ 40,000 

$ 61,768 

154.42% 

54.42% 

7.6% 

CCOP E 

85 

18 

1530 

$ 19,833 

$ 53,033 

267.40% 

167.40% 

12.0% 

CCOP F 

57 

23 

1311 

$ 29,000 

$ 45,442 

156.70% 

56.70% 

6.4% 

Totals 



15243 

$ 337,878 

$ 528,322 

156.36% 

56.12% 

3.70% 

Revenue per 
Knowledge Unit: 

$ 34.66 









Table 4. Revised KVA Analysis for USS NEVERSAIE (DDG-22). 

At first glance it appears that the decision to increase the number of executions 
was highly advantageous. The expenses, revenues and ROK to the system increased 
while the ROI changed from a negative to a positive value. The PM of the USS 
NEVERSAIE is undoubtedly feeling a sense of pride for a job well done. However upon 
closer review the PM is now faced with the situation in which the CCOP A system is now 
producing more output than it was previously. This may cause an adverse effect to other 
systems that were already operating at maximum capacity and are only able to process a 
select amount of data. This increase of output from CCOP A may cause a backlog and 
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additional expenses (e.g., processing time, labor costs) to the systems that require this 
data as input. Don’t use both e.g., and etc. in same sentence. 

This simple example demonstrates the concept that Program Managers for an 
individual IT portfolio need to be cognizant of the fact that their actions could impact the 
larger aggregated IT portfolio of the entire organization. The implications of a PM’s 
actions may have serious consequences (e.g., cost of storage the excess inventory, old 
inventory passing its expiration (if applicable), or costs associated with the production of 
inventory/output). While the PM of USS NEVERSAIL simply wanted all the ROIs in his 
IT portfolio to be positive, his actions caused a backlog of output and additional expenses 
to related systems. Bottom line is that PMs simply need to be aware that their IT 
portfolio management must have a holistic view with regards to the entire organization. 
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IV. 


CONCLUSIONS AND RECOMMENDATIONS 


A, CONCLUSIONS 

This thesis provides the steps that Program Managers and Program Executive 
Officers need to follow to effectively manage their IT portfolios. Using KVA analysis 
and ROI on IT systems provides the defensible metric of value. Applying this 
methodology to track the value-added of a technology in a core process provides 
leadership of any IT portfolio the ability to make sound investment decisions for any 
system. There are several reasons that using the KVA methodology to manage an IT 
investment portfolio is a viable solution to the directive issued by the DoD CIO. Here are 
some of the key benefits: 

• Financial: The economical benefit is the ability to evaluate the Return on 
Investment to determine whether or not the cost of the investment was money well spent. 
The Department of Defense spends billions of dollars every year on developing and 
acquiring new technology. Conducting a KVA analysis on the IT investment portfolios 
would provide greater insight into determining if this money was not spent frivolously. 

• Managerial: Another benefit that is closely related to economics is the ease 
of management. Using the KVA steps detailed in this thesis allows PMs and PEOs the 
ability to calculate the standard deviation (i.e., volatility) of an IT portfolio. For the 
purposes of this thesis: Standard Deviation = Volatility = Risk. Using the standard 
deviation the PM will be able to identify the volatility of each of the subprocesses within 
the portfolio. This data along with the ROI information can be an excellent managerial 
tool to identify the IT systems that have higher risk. Systems with a large amount of 
volatility tend to indicate a high level of instability and may require the PM to conduct 
further analysis. 

• Real Options: After conducting a thorough KVA analysis of an IT 
investment portfolio the PM now has a better understanding of how each subprocess 
performs historically over time. The PM will be able to allocate more resources (e.g., 
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time and money) to those areas that require greater attention. Additionally, the PM will 
be able to make real options decisions such as: expanding the system, upgrading the 
system or recommending the abandonment of the system. 

B, RECOMMENDATIONS 

Program Managers and Program Executive Officers should start using the 
concepts of Harry Markowitz’s Modern Portfolio Theory to manage their IT investment 
portfolios. This theory was designed for an investment portfolio of the stock market; 
however the principles can easily be tailored to work with any IT investment portfolio. 
Department of Defense PMs and PEOs must understand the importance of the underlying 
principles of MPT and the risks involved in order to properly manage any IT investment 
portfolio (i.e., large or small). 

This thesis has demonstrated the benefits (e.g., financial, managerial, real options) 
of utilizing the KVA methodology on a portfolio of IT investments. Therefore, the 
Department of Defense should implement a policy that mandates the use of the KVA 
steps provided in this thesis. These steps illustrate the detailed framework that should be 
followed in order to effectively manage any IT investment portfolio. This policy should 
be strictly enforced to ensure that all PMs and PEOs use the KVA methodology on all 
current and future IT investment portfolios. These KVA steps will provide the PMs and 
PEOs with the necessary toolset to be able to make informed IT investment decisions. 
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V. FUTURE RESEARCH AREAS 


A baseline has been set in this research for the value of applying the KVA 
methodology to a private and public sector IT investment portfolio. There is still further 
research that can be conducted using actual real-world data. True historical data could be 
collected which would provide a greater understanding of the economical and managerial 
benefits of KVA and ROT Additionally, the potential would exist for a more detailed 
analysis of the calculated data and available real options. 

While conducting this research, one of the fundamental assumptions made was 
that the individual subprocesses had already been fully optimized to the greatest extent 
possible. A KVA analysis of an IT portfolio in conjunction with a thorough examination 
of Business Process Re-Engineering (BPR) could be conducted in a future study. The 
key to BPR is for organizations to look at their business processes from a "clean slate" 
perspective and determine how they can best construct these processes to improve how 
they conduct business. Conducting further research using KVA methodology and BPR 
would provide further insight into not only managing an IT portfolio, but the benefits of 
optimizing each subprocess in order to produce its maximum output. 


35 



THIS PAGE INTENTIONALLY LEET BLANK 


36 



LIST OF REFERENCES 


Capital Asset Pricing Model, (n.d.). Retrieved February 24, 2009, from Wikipedia: 
http ://en .Wikipedia .org/wiki/C apital_as set_pricing_model. 

Cook, Glenn. (2008). Lecture 4: Evaluation of Output (KVA) [PowerPoint slides]. 
Retrieved December 15, 2008, from course IS4220, 4* Qtr/2008, Naval 
Postgraduate School, Monterey, CA. 

Department of Defense. (1997). Guide for Managing Information Technology as an 
Investment and Measuring Performance. Version 1.0. Prepared by Vector 
Research, Inc. 

Department of Defense. (2006a). Information technology portfolio management 
implementation. DODINST8115.02. 

Department of Defense. (2006b). DoD Chief Information Officer strategic plan. 
Version 1. 

Department of the Navy Information Technology Capital Planning Guide. (2001). 

Retrieved February 5, 2009, from http://www.doncio.navy.mil/download.aspx? 
attachid=493. 

Elder, J. & Westra, P. (2000). The reaction of security prices to tracking stocks 
announcements. Journal of Economics and Einance. 

Elton, E. J. & Gruber, M. J. (1979). Portfolio theory, 25 years after. New York, NY: 
North-Holland Publishing Co. 

Eabozzi, E. J., Markowitz, H. M., Kostovetsky, E. (2004). The theory and practice of 
investment management workbook: Step-by-step exercises and tests to help you 
master the theory and practice of investment management. Hoboken, NJ: John 
Wiley & Sons, Inc. 

Hagin, R. (1979). The Dow Jones-Irwin guide to modern portfolio theory. Homewood, 
IE: Dow Jones-Irwin. 

Housel, T. J. & Bell, A. H. (2001). Measuring and managing knowledge. New York, 
NY: McGraw-Hill Companies, Inc. 

Housel, T.J. & Kanevsky, V. (n.d.). Knowledge Value-Added (KVA) Methodology. 
Retrieved January 25, 2009, from 
http: //w w w. iec. org/online/tutorial s/kv a/index. html 


37 



Housel, T. J. & Mun, J. (2007). A primer on applying Monte Carlo simulation, Real 
Options analysis, Knowledge Value Added, forecasting, and portfolio 
optimization [White paper]. Retrieved February 23, 2009, from Naval 
Postgraduate School, Monterey, CA. 

Hubbard, D. W. (2007). How to measure anything. Hoboken, NJ: John Wiley & Sons, 
Inc. 

Lambeth, I. D. & Clapp, H. N. (2007). Using Knowledge Value Added (KVA) for 

evaluating cryptologic IT capabilities: Trial implementation. Monterey: Naval 
Postgraduate School. 

Markowitz, H. M. (1952). Portfolio selection. The Journal of Finance, 7(1), 77-91. 

Markowitz, H. M. & Perold, A. F. (1981). Portfolio analysis with factors and scenarios. 
The Journal of Finance, 36(4), Sll-Sll. 

Markowitz, H. M., Todd, G. P., Sharpe, W. F. (1987). Mean-variance analysis in 
portfolio choice and capital markets. Hoboken, NJ: John Wiley & Sons, Inc. 

Markowitz, H. M. (1991). Portfolio selection: Efficient diversification of investment. 
Hoboken, NJ: John Wiley & Sons, Inc. 

Measuring the Return on Knowledge. (2006). Retrieved January 25, 2009, from 

http://www.knowledgeboard.com/download/3887/Measuring-the-Return-on- 

Knowledge.pdf 

Mclnish, T. H. (2000). Capital Markets: A Global Perspective. Oxford, England: 
Blackwell Publishing. 

Mun, J. (2005). Real options analysis: Tools and techniques for valuing strategic 
investments and decision. Hoboken, NJ: John Wiley & Sons, Inc. 

Office of Management and Budget. (2000). Management of Federal Information 
Resources. Circular No. A-130. 

Rios, C. (2005). Return on investment analysis of information warfare systems. 
Monterey, CA: Naval Postgraduate School. 

Rios, C., Housel, T., & Mun, J. (2006). Integrated portfolio analysis [electronic 
resource] : Return on investment and real options analysis of intelligence 
information systems (cryptologic carry on program). Monterey, CA: Naval 
Postgraduate School. 

Sharpe, W. F. (1970). Portfolio theory and capital markets. New York, NY: McGraw- 
Hill Book Co. 


38 



Smith, K. V. (1971). Portfolio management: Theoretical and empirical studies of 
portfolio-making. San Francisco, CA: Holt, Rinehart and Winston, Inc. 

United States Code. (1996). Clinger-Cohen Act (Formerly Information Technology 
Management Reform Act). Division E of Pub L. 104-106. 


39 



THIS PAGE INTENTIONALLY LEET BLANK 


40 



INITIAL DISTRIBUTION LIST 


1. Defense Technical Information Center 
Ft. Belvoir, Virginia 

2. Dudley Knox Library 
Naval Postgraduate School 
Monterey, California 

3. Tom Housel 

Naval Postgraduate School 
Monterey, California 

4. Jonathan Mun 

Naval Postgraduate School 
Monterey, California 

5. Dan Boger 

Naval Postgraduate School 
Monterey, California 


41 



